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1. INTRODUCTION 


Young teenaged drivers have a considerably higher crash rate than any other age group, with 
new teenaged drivers having the highest crash rates of any group of drivers. While research has 
struggled to find clear evidence that traditional high school driver education programs have a 
positive impact on safe driving, the hope is that emerging and future driver education programs 
will build upon the lessons learned from the traditional approaches to driver education. As one 
example, some experts have recommended a multistage training approach in which the 
traditional training is supplemented by a carefully designed advanced training program. Such an 
approach is advocated by the American Driver and Traffic Safety Education Association 
(Robinson, 2001) as part of a graduated licensing system, in which “initial training of novice 
drivers will provide basic vehicle handling skills and the second training course will provide 
other safe driving skills, including enhanced decision making to reduce the risk of young 
drivers.” 


This project was a three-phase effort to evaluate the effectiveness of such a multistage program 
for Montana’s young drivers. Phase 1 included selection and recruitment of participants and 
development of training materials. Phase 2 concentrated on three major tasks—final preparation 
for training, collection of driving experience data, and presentation of the training to the teen 
drivers. Phase 3 was a follow-up longitudinal study of the driving experiences and safety records 
of the Phase 2 participants. 


During Phase 1, approximately 400 teenaged drivers who had completed high school driver 
education agreed to participate in the study. The drivers were randomly and evenly divided into a 
treatment group (case group) that received the defensive driving workshop and a control group 
that did not. 


During Phase 2, the young drivers in the case group completed a detailed questionnaire 
developed by the Montana Office of Public Instruction (OPI) concerning their driving experience 
in the time since completion of driver education classes. They then completed approximately 
nine hours of instruction, both in a classroom setting and behind the wheel. These activities were 
conducted at a driver training facility in Lewistown, Montana. More detail about these activities 
can be found in Kelly and Stanley (2006). The half of the teen drivers who were not drawn to 
take part in the training workshops were mailed survey forms that were identical to those 
completed by the students at Lewistown. Approximately 350 usable responses to the 
questionnaire were received from the two groups. 


During Phase 3, the driving experiences of the trained/case and non-trained/control drivers were 
followed for a period of four years. Using the OPI-developed written questionnaires mailed to 
each participant, reports of crash and violation histories of the participants were obtained. This 
report summarizes the work done on the project for the four-year period since the end of the 
defensive driving training workshops and includes analyses of safety-related data collected to 
date. 
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2. BACKGROUND 


2.1. The Experience of Young Drivers 


Each year, roadway crashes take the lives of approximately 40,000 people and seriously injure 
another three million in the United States (U. S. Department of Transportation, 2005). The cost 
of these crashes approaches $200 billion. 


Teenaged drivers have a considerably higher crash rate than any other age group. Figure 1 shows 
that drivers under the age of 20 have a crash rate four times that of the general driving population 
(Williams, 2003). New teenaged drivers have the highest crash rates of any group of drivers. The 
highest crash rate is experienced within two years of receiving the driving license. As expected, 
the crash rate decreases with driving experience and increased maturity. Research is needed to 
determine how to safely equip novice drivers with the important elements of experience before 
they encounter a need for it in an actual driving situation. Many novice driver crashes involve 
improper reactions to skids, panic stops, run-off-pavement, and other unusual situations 
unfamiliar to the young driver. Other crashes can partially be attributed to lifestyle issues such as 
risk-taking, risk-seeking, peer pressure and approval, distraction, and substance abuse. 


Crash Rate by Driver's Age 


Crashes per Million Miles 


15-19 20-24 25-29 30-34 35-39 40-44 45-49 50-54 55-59 60-64 65-69 70-74 75-79 80-84 
Driver Age 


Figure 1. Crash Rate by Driver's Age (from Williams, 2003). 


In the United States, most driving training is provided informally by parents or, more formally, 
in high-school-affiliated classes. These classes require numerous hours (typically 30) of 
classroom instruction on rules of the road, vehicle operation, and safety. The nascent drivers then 
spend several hours (typically six) behind the steering wheel driving in parking lots or in normal 
traffic on familiar streets. Only rarely do they experience circumstances in which the vehicle 
must be handled at its performance limits. 


Driver education in European countries is much more rigorous than that in the United States. 
Classroom training is presented on vehicle operating principles and basic maintenance. 
Typically, behind-the-wheel training provides more emphasis on the advanced aspects of vehicle 
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handling in potentially hazardous conditions. Classroom training provides more emphasis on 
cognitive factors such as risk perception. Also, the minimum age for driver licensing (typically 
18 years) is usually higher than the ages mandated by the states in the United States (Siegrist, 
2003). 


Several organizations in the United States offer training in advanced vehicle handling for novice 
drivers (Car Control, n.d.a.). Such training is intended to supplement basic driving classes and 
typically includes vehicle control on skid pads, obstacle avoidance, rapid deceleration braking, 
and maneuvering near the vehicle performance limits. While there is considerable anecdotal 
evidence that such training creates a more skilled and capable novice driver when added to the 
standard driver instruction, few systematic studies of its effect on the safety of young drivers 
have been completed. Where such studies have been done, results are contradictory and subject 
to interpretation and controversy. 


2.2. Research on Driver Training 


Many questions have been raised concerning the effectiveness of conventional young driver 
education programs. A former researcher at the Insurance Institute for Highway Safety, Williams 
(2003) declared these short-term programs to be unrealistic approaches to increasing the safety 
of young drivers. Conversely, a study conducted by the Oregon Department of Transportation 
(ODOT) and the Center for Applied Research found “significantly lower rates of convictions, 
suspensions and crashes” for those taking the driver education course versus those who learned 
through 50 hours of informal, supervised training (Triplett, 2005). 


International literature demonstrates little proof that formal driver instruction increases driver 
safety, yet arguably these programs have failed to adequately address age- and experience- 
related factors that contribute to a young driver’s increased risk of crashes. It is believed that 
such programs can be more effective if they are more empirically based, addressing the age- and 
experience-related factors (Mayhew and Simpson, 2002). Mayhew and Simpson state the reasons 
why formal instruction fails to reduce crashes: 


e Driver education/training fails to teach the knowledge and skills critical 
for safe driving. 


e Driver education does teach safety skills but students are not motivated to 
use them. 


e Driver education fosters overconfidence. 
e Driver education fails to adequately address lifestyle issues. 
e Driver education fails to tailor content to student needs. 


The well-known DeKalb driver education study, conducted in suburban Atlanta, was one of the 
first attempts to systematically validate the benefits of driver education (Stock et al., 1983). A 
cohort of 16,000 high school students was examined. The participants were divided into three 
groups based on the training they received, i.e., no training at all, a minimal curriculum of 20 
hours of training, or a Safe Performance Curriculum (SPC) of 70 hours of training. The SPC 
curriculum was based on a task analysis of required driver skills, but little information survives 
about how it was conducted. While there was some evidence that indicated training had a 
positive safety impact (e.g., increase in safety belt use) on teen driving during the first six 
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months after receiving training, there was no statistically significant differences in driving safety 
between the groups after the six months. One observation was that drivers in the “no-training” 
group delayed obtaining their driver's licenses as compared to drivers in the other groups. The 
methodology has generated considerable subsequent debate, especially concerning the possible 
lack of equivalency of the three groups and the inadequacy of the statistical model to show 
differences between them. Despite its limitations, this classic study has widely been considered 
the definitive evaluation and used as evidence to support the subsequent defunding of many high 
school driver education programs. 


Mayhew and Simpson (2002) completed a synthesis of research related to safety benefits of 
young driver training. They concluded that the major effect of traditional, school-affiliated driver 
education programs is to make licensing more readily available to younger drivers. They found 
no clear evidence that these traditional programs have a positive impact on safe driving. The 
authors recommended a multistage training approach in which the traditional training is later 
supplemented by a carefully designed advanced training program that: 


e Is focused on psychomotor, cognitive, and perceptual skills shown to be associated with 
high collision rates among young drivers; 


e Includes experiences demonstrating the value of safe driving practices; 
e Incorporates experiences that make the drivers more aware of their own limitations; 
e Uses techniques developed to address lifestyle and risk-taking behaviors; and 


e Recognizes that there are individual differences in skill levels and addresses specific skill 
deficiencies of the individual participants. 


Such an approach is advocated by the American Driver and Traffic Safety Education Association 
as part of a graduated licensing system in which, according to Robinson (2001), “Initial training 
of novice drivers will provide basic vehicle handling skills and the second training course will 
provide other safe driving skills, including enhanced decision making to reduce the risk of young 
drivers.” 


A study of over 400 graduates of an urban, East Coast course for young, previously licensed 
drivers reported that the graduates had 77 percent fewer crashes than their peers (Car Control, 
n.d.b.). That number, however, was probably inflated by a weak research design in which the 
more careful and highly motivated teens were self-selected into the training classes. A more 
carefully designed and controlled study was needed to validate those striking results. 


Skill-based training has created much discussion among driver education experts. Research has 
shown that skill-based strategies may produce overconfidence toward one’s own skills 
(Gregersen, 1996a). For example, Glad (1988) found that those taking part in skid training as a 
mandatory part of the training had an increase in slippery road crashes. Another study found that 
after the introduction of skid training into the education curriculum, higher rates of crashes 
occurred in slippery road conditions (Keskinene et al., 1992). It is believed that many skid 
training courses were based on maneuvering skills, leading to overconfidence. To counter this 
effect, it has been suggested that a distinction be made between training of skills and training of 
risk-awareness. Skill-based training concerns understanding vehicle control and maneuvering, 
while risk awareness is designed to increase knowledge, experience and recognition of dangers 
(Gregersen, 1996b; Sanders, 2003). A study on the effectiveness of skid-car training for teenage 
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novice drivers in Oregon found that females who received skid-car training had no change in 
crash rates, while the males appeared to have higher rates in the two years after training. 
However, it did appear that those receiving the training had relatively fewer slick-surface and 
rear-end collisions (Jones, 1995). 


The EU (European Union) ADVANCED Project (Sanders, 2003) developed several 
recommendations for post-license driver training. These recommendations were not objectively 
based but were based on the consensus of the researchers and investigators working in the area. 
The general recommendations include: 


e Courses should focus on the specific needs of participants and encourage them to 
improve their driving style and behavior. 


e Track-based driver courses should focus more on risk awareness than on maneuvering 
skills. 


e Comprehensive feedback and discussion sessions should be conducted after each on-road 
exercise. 


e To maintain individual attention, group size should not exceed 10 participants per 
instructor during track-based courses. 


e Training must be relevant to real-life situations and exercises and discussion should be 
related to real-life scenarios. 


e Overconfidence should be avoided; this is done by allowing students to fail (ie., hit 
obstacles, or lose full or temporary control of the vehicle). 


e Good client—trainer relations should be established to have the greatest influence on the 
participant throughout the course. 


Graduated licensing programs have been shown to significantly reduce young driver crashes and 
fatalities (McKnight and Peck, 2002). While these programs don't necessarily improve the skills 
of young drivers, they do reduce their miles of driving and their exposure to peer pressure and 
hazardous driving conditions during their early driving years (Fohr et al., 2005). During the 2005 
legislative session, a form of graduated licensing was instituted for Montana. Since 
implementation only began in 2006, it is too early to determine whether the expected benefits 
will materialize. 
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3. METHODOLOGY 


Approximately 400 teenaged drivers who had completed high school driver education were 
approached to participate in this study; a total of 347 agreed to participate. These drivers were 
randomly split into two groups of approximately equal size. One group received additional 
instruction in a defensive driving workshop, and the other group did not. Their subsequent 
driving safety experience over the following four-year period was tracked to assess whether the 
additional driver training has an impact on their safety. The large size of the sample and random 
assignment of the participants to the control and case groups allowed for this assessment of cause 
and effect to be more confidently made. 


3.1. Overview of Training 


The Montana OPI scheduled Lewistown Driver In-Vehicle Education (D.R.I.V.E.) facilities and 
instructors for 18 one-day sessions during the summer of 2005. Each day, 12 young drivers were 
scheduled to take the training workshops in Lewistown. The Western Transportation Institute 
(WTI) contracted with school bus providers for the Great Falls and Billings school districts to 
provide transportation to and from Lewistown. Students from Harlem were bused by their high 
school, which does not contract out its transportation services. Students from Lewistown and the 
surrounding communities provided their own transportation to the training facility. 


At the training facility, the young drivers completed a participant consent form and a detailed 
survey concerning their driving experience since completion of driver education classes. They 
then completed approximately nine hours of instruction in the classroom setting and behind the 
wheel. At the completion of the day's training, each student received a tailored “report card” 
concerning his or her driving performance, and exercises they could do on their own to improve 
it. The training is presented in more detail in Kelly and Stanley (2006). 


The half of the teen drivers who were not drawn to take part in the training workshops were 
mailed survey forms that were identical to those completed by the students at Lewistown, and 
they were asked to complete and return them to WTI. 


3.2. Driving History 


Students participated in a pretraining exercise in which, with no prior instruction, they were 
required to navigate two separate instructor-initiated skids in a specially equipped SkidMonster 
vehicle, and to demonstrate their vehicle positioning to the front and sides. After all students 
completed this exercise, they proceeded to the classroom where they filled out a human 
participant consent form. A detailed young driver survey (see Appendix A) completed by the 
participants on their driving experience since completing their driver education class was 
developed by the Montana OPI. The questionnaire was tailored to ask those questions that 
correlate highly to teen crash involvement, as determined from teen crash data. Information 
solicited by the survey included: 


e The number of hours per week they usually drive; 


e The number of passengers (and age classification) usually in the vehicle 
and how often they have passengers in their car; 


e Type of vehicle driven; 
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e Time of day they usually drive; 

e History of traffic citations and warnings; 
e History of near-miss crashes; 

e History of single-vehicle crashes; and 


e History of multiple-vehicle crashes. 


3.3. Classroom Instruction 


Upon completing the young driver survey and receiving the student folder of instructional 
materials the students who received the defensive driving workshop were taken to the Montana 
D.R.LV.E. classroom training facility. Training is described in detail by Kelly and Stanley 
(2006). Here training was done in two classroom periods (morning and afternoon) led by a 
classroom instructor. Both the morning and afternoon classroom instruction included PowerPoint 
presentations. The purpose of these presentations was to inform the students of driver readiness 
with reference to seat adjustment, mirrors, driver position, use of the ’dead pedal,” seat belts, 
balanced hand position on the wheel, and windows up. 


Brief overviews of the material presented in the classroom sessions are provided below. 


e Morning Classroom: “Montana Teen Class Phase I” presentation was provided to 
facilitate the lecture. Further demonstrations were provided on the effects of high speeds 
on losing control of the vehicle. This included using a small “Frisbee-type” saucer and 
matchbox cars to demonstrate the effects of speed on friction of the vehicle’s wheels. A 
slide-by-slide explanation of the Montana Teen Class Phase I PowerPoint presentation as 
lectured to the students was previously provided to MDT in the Training Materials. 


e Afternoon Classroom: “Montana Teen Class Phase II” presentation was provided to 
facilitate the lecture. No further demonstrations were provided. A_ slide-by-slide 
explanation of the Montana Teen Class Phase II PowerPoint presentation as lectured to 
the students was previously provided to MDT in the Training Materials. 


Integrated with the PowerPoint presentations were two interactive sessions using E-book 
activities, one in the morning and the other in the afternoon. Within the two E-book periods were 
embedded video clips demonstrating principles discussed. Provided in the E-book were 
interactive grids where students were to mark their answers to questions that were posed to them 
on principles discussed. Upon completion, students could check their answers with the provided 
answer sheets. 


A picture of the classroom instruction portion at the Montana D.R.I.V.E. training facility is 
provided in Figure 2. The classroom is a retired driver simulator trailer about 12 ft wide and 40 ft 
long. Three computers were set up to deliver the E-book training. A fourth computer was used to 
deliver the PowerPoint. Students sat in inactive simulator stations during the classroom 
instruction. 
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Figure 2. Instructional Classroom at Montana D.R.I.V.E. Training Facility. 


Additional classroom instruction was completed out-of-doors at the Montana D.R.I.V.E. training 
facility to allow students more arm and leg room to practice maneuvers as instructed. 


3.4. Behind-the-Wheel Instruction 


Behind-the-wheel instruction was done using three sedans equipped with SkidMonsters, a 
proprietary device used to teach vehicle control and skid recoveries. Two other sedans were 
equipped with levers to activate rear brakes. An additional regular sedan and a mid-1990s sport 
utility vehicle were used to teach reference points and off-road recovery. Figure 3 shows a 
vehicle equipped with the SkidMonster technology. 


Figure 3. Student Participating in SkidMonster Behind-the-Wheel Instruction. 


The two lever-equipped skid sedans were used in pre- and post-instruction skid assessments. The 
three SkidMonster vehicles were used to teach behaviors and skills related to 10 safe driving 
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habits documented by Mottola (2003). The driving track used was a paved “Monster Pad” that is 
200 ft by 600 ft. 


At the conclusion of the behind-the-wheel instruction and the post test on the wet skid pad, 
instructors took the students to the Monster pad and divided the group into two teams. They then 
had a “road rally” with each team member driving a SkidMonster-equipped car through the 
course in a timed event that included all aspects of training covered throughout the day. Rules 
and separate grade sheets were given beforehand so the drivers would know what to expect. 


Anecdotal reports from students who went through the workshop indicated that they felt more 
relaxed and confident about their driving ability. Instructors reported “they made great strides, 
showing improvement in the post-test of front/side limitations and skids; and, they also 
understood the importance of controlling the four-second danger zone and keeping the vehicle in 
balance.” Overall, the students gained valuable knowledge and skills with regard to driving, and 
became more confident in their ability to handle various driving situations. Analysis of the 
vehicle-handling scores, especially skid recovery, showed significant improvement in vehicle 
handling between the pre-testing and the post-testing. 


Using a paired t-test, an analysis of the students’ skid-car performance both before and after 
training was conducted to assess if the students improved according to the instructors scoring 
criteria. 


3.5. Collection of Driving-Related Data 


Figure 4 provides a timeline of the project’s data collection periods for Phase 3. 


Phase 3: Data Collection Timeline 


Summer 2005 August 2006 August )2007 August 2008 August)2009 August 2010 


Survey Data 
Collection 


Figure 4. Project Timeline. 


As previously discussed, during the initial year of the study, while training was being conducted, 
participants completed a written survey (Appendix A) regarding their driving experience that 
was developed by the Montana OPI. Participants who did not take part in the training were 
mailed the questionnaire during the same timeframe and reimbursed for their time in completing 
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it. Following the morning classroom and in-vehicle sessions, instructors assessed the students 
regarding their performance in the 10 safe driving behaviors emphasized throughout the training. 
Each category was rated by the instructor according to the student’s performance of specific 
actions. The ratings were scored by the instructor according to whether or not the specified safe 
driving behaviors were not followed, not always followed, or always followed. The behaviors 
and scoring metrics are shown in Table 1. 


Table 1. Instructor Report Card Scoring Metrics. 


Driving Behavior 


Instructor Scoring Metrics 


Shows Driver-Vehicle Readiness 


Always use safety belts and headlights 
Show correct seating position 

Show correction position and use of steering wheel 

Stay focused on driving task 


Accurately Positions Vehicle 


Keeps Car in Balance 


Use lane positions effectively 


Position right side to road edge with accuracy 
Position front bumper to intersection with accuracy 


Make smooth and effective starts and stops 

Get speed control before turning steering wheel 
Use controlled braking 

Use “transition pegs” while braking or accelerating 


Establishes Clear Path 


Search the “Target area” 

Evaluate intended path of travel 

See open space before accelerating 
Turn head before turning steering wheel 


Handles “LOS/POT Blockages” 


Controls Intersections 


Control four-second danger zone 


Detect LOS-POT blockages 
Check for escape/alternate path 
Get best speed control and lane position 


Search left, front, and right zones for “open” zones 
Detect LOS-POT blockages 

Control four-second danger zone 

Control two-second “Point of No Return” 


Controls Rear Zone 


Check mirrors when braking 
Check blind spots before moving into another lane 


Controls Curves 


Use lane positions correctly 

Reduce speed within four-second danger zone 

Use “transition peg” to determine when to accelerate 
Look through curve for “Open/Closed POT” 


Controls Vehicle in Front 


“Close in” on slower moving vehicles gradually 
Keep four-second following time 
Stop behind vehicle to see rear tires 


Controls Emergency Situations 


Detect “off target” skid conditions 
Take corrective actions without delay 


Point head and eyes toward target area 
Effectively use “vehicle controls” 


Western Transportation Institute 


Page 10 


Effects of Defensive Vehicle Handling Training: Phase 3 Methodology 


At the end of the first year after the defensive driver training was conducted, and each year 
thereafter, a survey covering the prior year was mailed to the study participants. These surveys 
were identical to the one they were asked to complete prior to the training course. They were 
reimbursed with a $10 payment for their time in completing and returning the survey. Of the 347 
surveys mailed out for the first survey period (2006), 284 participants returned the surveys (82 
percent return rate). For the second survey period (2007), 347 surveys were mailed again; 270 
participants returned the surveys (78 percent return rate). In September 2008, surveys were sent 
to the 347 participants using the addresses provided during their training. Ninety-nine 
participants responded to the initial mailing, and an additional 63 responded following a 
reminder/secondary mailing. Of the 161 that responded (47 percent response rate), 23 
participants chose to use the online survey response option. Of the 347 original project 
participants, mailings to 38 participants were returned as undeliverable. Specific response data is 
shown in Table 2 for both the case and control groups. 


Table 2. Survey Response Rates. 


Survey Response Rates 
Case Group Control Group 
N=165 N=182 

n % n % 
Year | 146 88% 138 76% 
Year 2 125 76% 145 80% 
Year 3 76 46% 85 47% 
Year 4 82 50% 85 4AT% 


In addition to the survey, participants’ Department of Motor Vehicle (DMV) records were 
obtained using the Montana Department of Justice Motor Vehicle Division’s online driver 
history record service (https://app.mt.gov/dojdrs/ ). The analysis of the participants’ driving 
records in the study serves the purpose of providing a more objective form of data collection 
compared to that of the self-reported survey data. Data collected through both methods were 
analyzed, as studies have indicated that DMV data may slightly under-represent driver behaviors 
since the records do not contain information regarding unreported incidents or all unsafe driving 
behaviors (Schultheis et al., 2002). DMV records were available for 295 participants. The other 
52 participants were not found in the DMV database, either because the participants never listed 
their Montana driver’s license number on their surveys or because they had since moved out of 
state. DMV data availability is shown in Table 3. When the DMV data was reexamined in 2009, 
records were available for 309 participants. The increase was due to the participants including 
information on returned surveys that was required to search the DMV records, but that was 
previously missing or illegible on handwritten surveys (e.g., correct date of birth, drivers license 
numbers). 
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Table 3. DMV Data Availability. 


DMV Data Availability 
Case Group Control Group 
N=165 N=182 

n % n % 
Year 1 138 84% 157 86% 
Year 2 138 84% 157 86% 
Year 3 138 84% 157 86% 
Year 4 144 87% 165 91% 


3.6. Develop Database for Archiving and Analysis 


During Year One, all survey data were obtained in written form on paper. In order to support 
archiving and analyzing the data, a database was developed to record all of the data obtained in 
2005 through 2009. 


Data provided by the participants during Year One (August 2005—August 2006) and subsequent 
years were transferred from their paper forms into a database. The database contained names and 
updated mailing addresses of all participants and the driving experience data they reported each 
year. It was suitable for preparing mailing labels for contacting participants as well as archiving 
and analyzing data. As written forms were received from participants, the data they submitted 
was entered for analysis. 


3.7. Test and Control Group Baseline 


Selected information collected from the participants on their pre-program driving history was 
analyzed to document their experiences in their first months after high school driver training and 
to further ensure that the Case Group (those who had received the training) and the Control 
Group were equivalent on traffic-safety-relevant attributes before the defensive training 
workshops were presented. During the initial driving period, approximately 18 percent of the 
drivers in both groups reported experiencing crashes. Approximately 17 percent of the Case 
Group and 19 percent of the Control Group were given traffic citations or warnings. These data 
suggest that by using a random group assignment, the groups were equivalent before the training 
workshops. 


3.8.  Post-intervention Data and Methods 


Descriptive statistics were summarized for 2006 through 2009 on safety-relevant data. The 
dependent variables, termed “incidents,” reported by each driver were classified into five main 
categories: citations, near-miss collisions, single-vehicle collisions, multiple-vehicle collisions, 
and total collisions. Citations included moving warnings, moving violations, parking tickets, and 
minors in possession (MIP). Near-miss collisions were incidents that did not involve an actual 
collision; rather the collision was avoided by a narrow margin. Single-vehicle incidents were 
events that involved only the driver’s vehicle, whereas multiple-vehicle incidents involved two 
or more vehicles. The total number of collisions included both single- and multiple-vehicle 
incidents. Detailed descriptions of the dependent variables collected in this study are presented in 
Table 4. 
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Table 4. Description of Dependent Variables (shaded variables are survey based). 


Survey Citations 


The number of moving warnings, moving violations, parking 
tickets, and minors in possession (MIP) 


DMV Citations 


The number of citations each participant received according to the 
Montana DMV 


DMV Driving-Related 
Citations 


The number of driving-behavior-related citations each participant 
received according to the Montana DMV 


Survey Near-Miss 
Incidents 


The number of incidents that did not occur but were avoided by a 
narrow margin 


Survey Single-Vehicle 
Incidents 


The number of incidents that involved only the driver’s vehicle 


Survey Multiple-Vehicle 
Incidents 


The number of incidents that involved two or more vehicles 


Survey Total Collisions 


The sum of single-vehicle incidents and multiple-vehicle 
incidents (see above). 


Exposure The number of hours per week each participant reported driving 
ad] uses Sitons Citations Received/Number of Hours Driven/Month 

Received 

Adj usted pi ive auons DMV Citations/Number of Hours Driven/Month 

Received 

Adjusted DMV Citations, | DMV Citations—Driving Related/Number of Hours 

Driving Related Driven/Month 


Adjusted Near-Miss 
Incidents 


Near-Miss Incidents/Number of Hours Driven/Month 


Adjusted Single-Vehicle 
Incidents 


Single-Vehicle Incidents/Number of Hours Driven/Month 


Adjusted Multiple-Vehicle 
Incidents 


Adjusted Total Collisions 


Multiple-Vehicle Incidents/Number of Hours Driven/Month 


Total Collisions/Number of Hours Driven/Month. 


Pre-training Score 


Instructor scores (for Case Group only) on defensive maneuvers 
prior to being trained on the skid cars 


Post-training Score 


Instructor scores (for Case Group only) on defensive maneuvers 
after being trained on the skid cars 
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3.9. Statistical Methods Used for Data Analysis 


In order to determine the effectiveness of the training program, inferential statistics were used to 
test for the statistical significance of group membership and performance year. Due to the non- 
parametric nature of count data, a Chi-squared analysis on frequency counts was used to 
determine statistically significant relationships for each incident type (citation, near miss, single 
vehicle, multiple vehicle, and total collisions) within each group over time, as well as within 
each time period between each group. When a statistically significant difference was calculated 
the respective Chi-square and p-values were reported. For the parametric data, e.g., exposure- 
based data, an Analysis of Variance (ANOVA) was used to determine where statistically 
significant (p<0.05) differences were present. When a statistically significant difference was 
calculated the respective F-value and p-values were reported. 


To determine the effect size of the Case Group versus the Control Group an odds ratio was 
calculated. The odds ratio is defined as the ratio of the odds of an incident occurring in one group 
to the odds of it occurring in another group, or a sample-based estimate of that ratio. In this case, 
the odds ratio was generated to represent the odds of an incident, e.g., receiving a citation, in the 
Control Group to the odds of receiving a citation in the Case Group. The odds ratio calculation is 
shown in Equation 1. 


p/(l—p)  p(l—q) 
qg/(l1—q) q(1-p) 


Equation 1 


Where: 


p = probability of the event in each of the groups (untrained, or control group) 
q = probability of the event in each of the groups (trained, or case group) 


An odds ratio of one indicates that the incident under question is equally likely to occur in both 
the Case and Control Group. When the odds ratio is greater than one, this indicates that the 
incident under question is more likely to occur in the Control Group. If the odds ratio is less than 
one, this indicates that the incident is less likely to occur in the Control Group. 


The data provided through surveys were evaluated within each group (Case or Control) over 
time, as well as within each survey collection period (2005—2006, 2006-2007, 2007-2008, and 
2008-2009). The metrics that were evaluated independently of exposure were tested using Chi- 
squared analysis between groups using counts of participants who reported at least one incident, 
or who reported no incidents for each category. This method is preferred as the Case and Control 
Groups have unequal sample sizes of survey responses in every survey period. While testing the 
survey data to observe change over time, Chi-squared analysis was again used because there 
were unequal sample sizes from each group responding in each year. 


An exposure analysis was conducted to determine if those participants who drove more hours per 
week experienced more incidents. In addition, this parametric data was analyzed using a repeated 
measures ANOVA to determine the main effects of time. In other words, over time does the 
number of incidents significantly increase or decrease? Only those participants who responded to 
the survey for all four response periods were used in the repeated measures ANOVA. There were 
51 participants in the Control Group and 53 in the Case Group who consistently responded to the 
survey in years 2006 through 2009. 
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4. RESULTS 
4.1. Survey Data 


4.1.1. Summary Statistics by Year 


The descriptive statistics and calculated odds ratio for the number of incidents for the Case and 
Control Groups in 2006 through 2009 for each dependent variable are provided in Table 5. In 
reference to the 2006 data, citations, multiple collisions, and total collisions are more likely to 
occur in the Control Group, whereas near misses are more likely to occur in the Case Group. In 
2007, the Control Group is more likely to receive citations, be involved in near misses, and 
experience multiple collisions, while those in the Case Group are more likely to be involved in 
single collisions and total collisions. In 2008, the Control Group is more likely to be involved in 
a near miss, while those in the Case Group are more likely to be involved in single, multiple and 
total collisions. Viewed through time, the only consistent apparent trend is a shift in the relative 
likelihood of a collision away from the Control Group in 2006 toward the Case Group in 2009. 
Clearly, when viewed in this manner, the results are mixed in terms of the training program’s 
effectiveness, and thus a more thorough statistical analysis was conducted, with the results 
presented below. 


Table 5. Summary of Survey Data. 
2006 Survey Data 


Case Participants Control Participants 
No 
No Incident | Incident | Incident | Incident | Case | Control | Difference | Odds 
Citations 118 29 97 41 20% | 30% -10% 1.7 
Near Miss 84 63 89 49 43% | 36% 7% 0.7 
Sel 128 19 120 18 13% | 13% 0% 1 
Vehicle 
MOOS) 136 21 112 26 | 14% | 19% 5% | 14 
Vehicle 
Total 112 35 101 a7 24% | 27% -3% 2 
2007 Survey Data 
Case Participants Control Participants 
No 
No Incident | Incident | Incident | Incident | Case | Control | Difference | Odds 
Citations 86 39 80 65 31% | 45% -14% 1.8 
Near Miss 89 36 95 50 29% | 34% -6% 1.3 
eek 109 16 132 13 13% | 9% 4% 0.7 
Vehicle 
ote ed 26 107 38 | 21% | 26% | -5% | 14 
Vehicle 
Total 85 40 102 43 32% | 30% 2% 0.9 
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2008 Survey Data 


Results 


Control 
Case Participants Participants 
No No 
Incident | Incident | Incident | Incident | Case | Control | Difference | Odds 

Citations 86 39 80 65 31% | 45% -14% 1.8 
Near Miss 89 36 95 50 29% | 34% -6% 1 
Single 109 16 132 13 13% | 9% 4% 0.7 

Vehicle 
MBC”. - <0 26 | 107 | 38 | 21% | 26% | -s% | 14 

Vehicle 
Total 85 40 102 43 32% | 30% 2% 0.9 

2009 Survey Data 
Control 
Case Participants Participants 
No No 
Incident | Incident | Incident | Incident | Case | Control | Difference | Odds 

Citations 65 ia 70 15 21% 18% 3% 0.8 
Near Miss 66 16 67 18 20% 21% -1% 1.1 
ae 75 6 82 3 ™m | 4% 3% 0.5 
Mune) 93 9 75 10 | 11% | 12% | -1% | 44 

Vehicle 
Total 68 14 72 13 17% 15% 2% 0.9 

4.1.2. | Exposure —- Number of Hours Driven per Week 


Figure 5 provides the self-reported number of hours driven per week by the participants. In year 
2006 and 2007, the number of hours exposed to driving (hours driven/week) was not 
significantly different between the Case and the Control Group. In 2008, the Case Group 
reported spending significantly (p=0.019) more time behind the wheel per week than the Control 
Group—the Case Group reported spending roughly 8.5 hours per week and the Control Group 
reported 6.4 hours per week. One may assume the more exposure to driving the greater 
likelihood of a safety breach. Therefore, subsequent data analysis was conducted to account for 
driving exposure during that time period. No significant difference between the Case and Control 
Groups was found for 2009 driving exposure. 
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Driving Exposure by Year 
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Figure 5. Hours Driven per Week. 


4.1.3. Citations 


A summary of the number of self-reported citations for each year is provided in Table 6. 
Analysis of the 2006 survey data indicated that there was a statistically significant difference 
(°=3.828, p=0.05) in the number of citations between the Case and Control Group. Those in the 
Case Group reported fewer citations (20 percent) than those in the Control Group (30 percent). 
Similarly, in 2007 a statistically significant difference (y’=5.33, p=0.02) was determined between 
the Case and Control group. Those in the Case Group reported receiving fewer citations (31 
percent) than those in the Control Group (45 percent). However, in 2008 the number of citations 
is almost identical between the groups, with an odds ratio of one. There is additionally no 
significant difference in reported citations between the groups in 2009. This result may indicate 
that the effects of the training are no longer noticeable three years beyond training. 


Table 6. Reported Citations. 


Control 
Case Participants Participants 
None | Incidents | None _ Incidents Case | Control | Difference | Odds 
2006 | 118 29 97 41 20% 30% -10% Ley 
2007 | 86 39 80 65 31% 45% -14% 1.8 
2008 | 53 22 62 25 30% 29% 1% 1.0 
2009 | 65 Li 70 iS) 21% 18% 3% 0.8 


From 2006 to 2007, the number of students who received one or more citations increased (Figure 
6) for both the Case and Control Groups. In the Case Group, 20 percent reported receiving one or 
more citations in 2006, increasing significantly (y°=4.742, p=0.029) to 31 percent in 2007. The 
Control Group demonstrated significantly (y’=6.898, p=0.009) higher percentages of citations in 
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2007 than in 2006—30 percent reported one or more citations in 2006, and 45 percent in 2007. 
Following a peak in 2007, the 2008 and 2009 data demonstrate a downward trend for both Case 
and Control Groups. The Case Group shows a marginally significant decrease from 2007 to 2009 
(°=2.75, p=0.097). The Control Group shows a significant decrease in citation percentage from 
2007 to 2008 (x’?=5.93, p=0.015), and a marginally significant decrease from 2008 to 2009 
(°=2.96, p=0.085). 


Overall, in terms of the number of citations, it appears the training is more effective than no 
training from 2006 to 2007. However, both groups witnessed an increase in the number of 
citations from 2006 to 2007, but both decreased to their lowest levels in the fourth year. When 
adjusted for driving exposure, the number of reported citations was not significantly different 
between the groups during any survey year. 


Percentage of Participants Reporting One or More Citations, by Year 
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Figure 6. Percent Reporting Citations by Year. 


4.1.4. Near-Miss Incidents 


The survey data on near misses is summarized in Table 7 and Figure 7. In terms of the number of 
one or more near-miss incidents, no significant differences were found between the Case Group 
and the Control Group in any year. Moreover, when adjusting for driving exposure, the number 
of near-miss incidents was not significantly different between the groups during any survey year. 
Examining each group over time demonstrated a statistical significance with the Case Group 
having fewer participants reporting near-miss incidents each year (7°=27.4, p<0.0001). The 
Control Group also showed a downward trend, showing significant differences between years 
2006 and 2009 (y7=5.14, p=0.023). Near-miss incidents for 2007 and 2008 support the 
downward trend, but the near miss involvement for years 2007 and 2008 was not large enough to 
show a statistically significant decline. 
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Table 7. Reported Near-Miss Incidents. 


Control 
Case Participants Participants 
None | Incidents | None | Incidents Case | Control | Difference | Odds 
2006 | 64 63 89 49 43% 36% 7% 0.7 
2007 | 89 36 95 50 39% 34% -6% 1.3 
2008 | 58 17 62 24 23% 28% -5% 1.3 
2009 | 66 16 67 18 20% 21% -1% 1.1 


Percentage of Participants Reporting One or More Near-Miss 
Incident, by Year 
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Figure 7. Percentage of Participants Reporting One or More Near-Miss Incident. 


4.1.5. | Single-Vehicle Collisions 


Table 8 and Figure 8 provide the single-vehicle collision data reported for each survey period. 
The difference between Case and Control Groups for participants reporting one or more single- 
vehicle collisions was found to be significant (X’=3.89, p=0.049) during the 2008 period. The 
Case Group reported significantly more single-vehicle collisions (14 percent) than the Control 
Group (5 percent). However, when adjusted for driving exposure, the number of single-vehicle 
collisions was not significantly different between the groups during any survey year. The Control 
Group experienced a statistically significant (X°=8.22, p=0.042) decrease in the number of 
reported single-vehicle collisions from 2006 to 2009. The Case Group showed a decrease in the 
number of reported single-vehicle collisions from 2008 to 2009, although this decrease was not 
statistically significant. 
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Table 8. Reported Single-Vehicle Collisions 


Control 
Case Participants Participants 
None | Incidents | None | Incidents Case | Control | Difference | Odds 
2006 | 128 19 120 18 13% 13% 0% 1.0 
2007 | 109 16 132 13 13% 9% 4% 0.7 
2008 | 65 10 83 4 14% 5% 9% 0.3 
2009 | 75 6 82 3 7% 4% 3% 0.5 


Percentage of Participants Reporting One or More Single-Vehicle 
Collision, by Year 
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Figure 8. Percentage of Participants Reporting One or More Single-Vehicle Collision. 


4.1.6. Multiple-Vehicle Collisions 


The difference between Case and Control Groups in the self-reported number of one or more 
multiple-vehicle collisions was determined to be statistically insignificant over the four-year 
survey period. Moreover, when adjusting for driving exposure, the number of reported multiple- 
vehicle collisions was not significantly different between the groups during any survey year. 
Observing the data from 2006 to 2007, similar to the citation findings, those in the Case Group 
were involved in consistently fewer multiple-vehicle collisions than the Control Group. 
However, in 2008 the case participants reported having more multiple-vehicle collisions (12 
percent) than the Control Group (8 percent). The change over time for the Case Group showed a 
marginally significant decrease in the number of participants reporting multiple-vehicle 
collisions between 2007 and 2009 (X=4.64, p=0.098). Although the change over time for the 
Case Group was not large enough to be statistically significant, the Control Group showed 
significant variability (X°=15.3, p=0.002) from 2006 to 2009. The summarized findings are 
presented in Table 9 and Figure 9. 
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Table 9. Reported Multiple-Vehicle Collisions. 


Control 
Case Participants Participants 
None | Incidents | None | Incidents Case | Control | Difference | Odds 
2006 | 126 21 112 26 14% 19% -5% 1.4 
2007} 99 26 107 38 21% 26% -5% 1.4 
2008 | 66 9 80 7 12% 8% 4% 0.6 
2009 | 73 9 75 10 11% 12% -1% 1.1 


Percentage of Participants Reporting One or More Multiple-Vehicle 
Collisions, by Year 
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Figure 9. Percentage of Participants Reporting One or More Multiple-Vehicle Collisions. 


4.1.7. Total Vehicle Collisions 


The summarized data for total vehicle collisions is shown in Table 10 and Figure 10. The 
number of study participants who were involved in a collision, either single-vehicle or multiple- 
vehicle, was significantly (X°=4.54, p=0.033) different between the Case and Control Groups for 
the 2008 period. The Case Group reported significantly more total vehicle collisions (21 percent) 
compared to the Control Group (11 percent). However, when adjusted for driving exposure, the 
total number of vehicle collisions was not significantly different between the groups during any 
survey year. Examining the change over time, both groups showed an increase in the number of 
total vehicle collisions between 2006 and 2007. The Case Group showed a significant decrease 
from 2007 to 2009 (X?=6.58, p=0.037). The Control Group showed a decrease from 2007 to 
2008 (X?=10.2, p=0.001) followed by an insignificant increase from 2008 to 2009. 
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Table 10. Reported Total Vehicle Collisions. 


Control 
Participants 


Case Participants 


None | Incidents | None | Incidents Control Difference 
2006} 112 aS 101 37 24% 27% -3% 1.2 
2007 | 85 40 102 43 32% 30% 2% 0.9 
2008 | 59 16 77 10 21% 11% 10% 0.4 
2009 | 68 14 72 13 17% 15% 2% 0.9 


Percentage of Participants Reporting One or More Total Vehicle 
Collisions, by Year 


™ Case 


@ Control 


Figure 10. Percentage of Participants Reporting One or More Total Vehicle Collisions. 


4.1.8. Limitations of Aggregating Survey Data by Collision and Near- 
Miss Type 


It should be noted that this analysis by specific types of collisions and near misses relied on self- 
reported survey information that lacked a consistent reporting format for the young drivers to 
follow. Collision and near-miss descriptions were often vague and seldom included adequate 
information to determine collision type, region of impact, striking vehicle, or the at-fault driver. 
Therefore, results of that analysis may not be sufficient to draw prescriptive conclusions. 


It should also be noted that in 2004 the National Highway Traffic Safety Administration’s 
(NHTSA) Fatality Accident Reporting System (FARS) reported that 75 percent of fatal crashes 
among drivers between the ages of 16 and 19 were due to driver error, 35 percent involved the 
driver speeding, and nearly 50 percent of the incidents involved single-vehicle collisions. In the 
participants’ self-reported crash descriptions, few definitively identified the at-fault driver or 
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admitted to driver error, and the posted or actual traveling speed at the location of the incident 
was not given. 


Specific to the near-miss information in this study, only 20 percent of near-miss reports in 2006 
provided enough information to determine what would have been the striking vehicle if the crash 
occurred, 23 percent in 2007, 19 percent in 2008, and 20 percent of near-miss reports in 2009. 
The lack of consistent data regarding each crash or near-miss event negates meaningful statistical 
analysis by crash type. Because of these stated limitations, future studies seeking to utilize 
survey data should consider using a consistent categorical measure based upon standardized 
vehicle crash databases, such as the FARS and/or NHTSA’s General Estimates System. 


4.2. DMV Data 


The DMV records provided every official citation received (in Montana) by participants within 
the survey response periods. The citation codes were classified into “driving related” or “other” 
offenses, as shown in Table 11. Driving-related citations were grouped together so that their 
analysis could provide insight about driving performance, and the non-driving-related citations 
were grouped to demonstrate an indication of driver attitude or general driver safety culture. The 
citation groups were evaluated in each year between groups (Case and Control) and again within 
each group over time. No participants were cited for being involved in a crash and only the codes 
listed in Table 11 were found among the participant records. Thus, there appear to be some 
discrepancies between the self-reported and DMV-based assessment of participant driving 
experiences. It was discovered that the discrepancy was due to the reporting of parking tickets in 
the self-reported citations, which is not reported within the DMV database of citations. 


Table 11. Montana DMV Citation Codes Reported from Participants’ Records. 


Driving-Related Citation Codes 


Description 


Stop Sign Violation 

Failure to Obey Red (Stop) Traffic Signal 

Flashing Signal Violation (Red or Yellow) 

Stop Violation: Emerging From Alley, Private Road, Building, or Driveway 
Following too Closely 

Changing Lanes when Unsafe To Do So 

Careless Driving 

Reckless Driving 

Failure to Yield When Entering Hwy From Private Road or Driveway 
Failure to Yield to Vehicle on Right 

Failure to Yield to Vehicle Making Left Turn 

Unsafe Left Turn Across Lane Marked With Two Yellow Lines 
Improper Approach When Making Right Turn 

Interfering With Traffic While Backing 

Speeding Interstate Urban - Exceed Night Limit Of 65 
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Other Citation Codes 


pg Description 
Driving With BAC > 0.08 

Driving Under Influence of Alcohol 

Failure to Notify Owner Following Unattended Vehicle Damage 
Failure to Report Accident 

No Motorcycle Endorsement 


Driving Without a License 


Failure to Carry Proof of Insurance 

Improper Use of Amber Light - Use Only For Funeral 

Basic Rule - Reasonable And Prudent 

Leaving Unattended Vehicle Without Setting Brake 
Operating an Improperly Registered Vehicle 

Operating a Vehicle with License Plates Obstructed to View 


4.2.1. DMV Driving-Related Citations 


A summary of the DMV driving-related citation data is found in Table 12. The number of 
driving-related citations was not significantly different between the Case Group and the Control 
Group during any survey year. Additionally, when adjusted for driving exposure, differences 
between the number of DMV driving-related citations were also found to be insignificant 
between the groups during any survey year. 


Table 12. Number of DMV Driving-Related Citations 


Control 
Case Participants Participants 

None | Incidents | None | Incidents 
117 21 133 24 
123 15 147 10 


Case | Control | Difference | Odds 
15% 15% 0% 1.0 

6% 12% 
7% 10% -3% 1.6 
13% 12% 1% (iD 


2006 
2007 
2008 
2009 


4.2.2. DMV Non-Driving-Related Citations 


Table 13 summarizes the findings for the non-driving-related citations. No significant differences 
were found between the Case Group and the Control Group in the number of non-driving-related 
citations reported. Additionally, when adjusted for driving exposure, differences in the number 
of DMV non-driving-related citations were also found to be insignificant between the groups 
during any survey year. 
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None | Incidents | None | Incidents Control | Differences 
2006 | 133 5 153 4 4% 3% 1% 0.7 
2007 | 133 5 147 10 4% 6% -2% 1.8 
2008 | 133 5 152 5 4% 3% 1% 0.9 
2009 | 136 8 152 13 6% 8% -2% 1.4 
4.3. Gender Effects 


Case Participants 


Table 13. Number of DMV Non-Driving-Related Citations. 


Control 
Participants 


Results 


The data was also studied examining the difference in response variables between groups 
consisting of male and female participants to look for potential gender effects on driver behavior 
and history. Males showed higher exposure in number of hours driven per week in all years, and 
a higher percentage of males reported one or more citation (both self-reported and DMV- 
reported) in all years. Males also have higher DMV-reported citations than females after 
adjusting for driving exposure. Males show higher percentage of involvement in single-vehicle 
collisions than females in any year. Females show higher percentage of involvement in multiple- 
vehicle collisions than males in any year. A summary of this data is shown Table 14 and more 
thorough data analysis is presented below. 
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Citations 
DMV 
Citations 
Near Miss 
Single 
Vehicle 
Multiple 
Vehicle 
Total 


Citations 
DMV 
Citations 
Near Miss 
Single 
Vehicle 
Multiple 
Vehicle 
Total 


Citations 
DMV 
Citations 
Near Miss 
Single 
Vehicle 
Multiple 
Vehicle 
Total 


Table 14. Male and Female Response. 
2006 Survey Data 


Results 


Male Female 
No 
No Incident | Incident | Incident | Incident M F Difference | Odds 
98 44 116 26 31% | 18% 13% 2.0 
113 37 129 16 25% | 11% 14% 2.6 
83 59 89 53 42% | 37% 5% 1.2 
117 25 130 12 18% 8% 10% 2.3 
120 22 117 25 16% | 17% 1% 0.9 
101 41 108 34 29% | 24% 5% 1.3 
2007 Survey Data 
Male Female 
No 
No Incident | Incident | Incident | Incident M F Difference | Odds 
72 63 94 41 47% | 30% 17% 2.0 
132 18 130 15 12% | 10% 2% 1.2 
95 40 89 46 30% | 34% -4% 0.8 
117 18 124 11 13% 8% 5% 1.7 
104 31 103 32 23% | 24% -1% 1.0 
87 48 94 42 36% | 31% 5% 1.2 
2008 Survey Data 
Male Female 
No 
No Incident | Incident | Incident | Incident M F Difference | Odds 
57 26 56 22 31% | 28% 3% 1.1 
127 23 133 12 15% 8% T% 2.0 
66 17 57 21 21% | 27% -6% 0.7 
75 8 72 6 10% 8% 2% 1.3 
75 8 70 8 10% | 10% 0% 0.9 
70 16 65 13 19% | 17% 2% 1.1 
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2009 Survey Data 


Results 


Male Female 
No 
No Incident | Incident | Incident | Incident M F Difference | Odds 
Citations 58 15 7 17 21% | 18% 3% 11 
ie 124 Ey) 132 21 21% | 14% 1%, 1.6 
Near Miss 57 16 16 18 22% | 19% 3% isl 
ee 66 6 91 3 8% | 3% 5% 2.8 
er 66 qf 82 12 10% | 13% -3% 0.7 
Total 60 15 79 15 20% | 16% 4% 13 


Males show higher exposure for driving time than females in all years; the higher male exposure 
is significant in 2006 (t=4.33, p=0.0001), 2008 (t=3.86, p=0.0002), and 2009 (t=5.63, p=0.0001). 
A summary of this data is shown in Table 15. 


Table 15. Driving Exposure by Gender. 


Male Female 
Average Average 
Number | (hours driven per week) Number _| (hours driven per week) 
Year | 145 8.31 143 7.10 
Year 2 136 9.19 135 8.82 
Year 3 83 8.08 79 6.63 
Year 4 73 9.84 94 7.63 


Males show a higher percentage of one or more self-reported citations than females in all years, 
significant only in 2006 (X°=6.14, p=0.013) and 2007 157, p=0.006). 


A higher percentage of males have one or more DMV-reported citations than females in all 
years, significant in 2006 (X*=9.3, p=0.002), and marginally significant in 2008 (X*=3.51, 
p=0.061). The average number of DMV citations per male (u=0.32) is significantly higher than 
the average number of DMV citations per female (u=0.11) in 2006 (t=2.63, p=0.009) and 
marginally significant in 2008 (Umate=0.25, LUfemale=0.09, t=1.85, p=0.064). When examined 
adjusting for exposure, males still have a higher number of DMV citations per subject, although 
the significance is not achieved. 


Males show a higher percentage of involvement in one or more single-vehicle collisions and total 
vehicle collisions in each year, although this higher rate of involvement is insignificant in any 
year. Females show higher percentage of involvement in one or more multiple-vehicle collisions, 
insignificant in any year. 


4.4. 


While not envisioned as part of the original project, some preliminary analyses were also 
completed at the end of this study on two specific elements of the training program, namely, the 
skid car training and the report card exercise. These analyses and their results are described 
below. 


Evaluation of Specific Elements of the Training Program 
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4.4.1. Skid Car Training Scores 


Each trainee participated in two skid-car trials at 30 mph, with no instruction given. The ratings 
were based on a scale of | to 5 (1 = Failure, wrong actions, no actions; 2 = Poor, correct action 
too late; 3 = Fair, corrected with difficulty; 4 = Good, slight delay, correct action; 5 = Excellent, 
no delay, correct action). Two trials were measured and scored for each pre- and post-training 
test. The analysis of the instructor’s evaluation of the participant’s skid car pre- and post-training 
exercises found that students received significantly (p<0.0001) higher scores post-training than 
pre-training (Figure 11). 


Pre- and Post-Skid-Training Instructor Scores 
(Case Group Only) 


© Trial 1 


Trial 2 


Instructor Scores 
(5 being optimal performance) 
N 


Pre-Training Post-Training 


Figure 11. Pre- and Post-Skid-Training Instructor Scores. 


4.4.2. Report Card Data 


Immediately upon completion of the training provided at the beginning of this program to the 
participants in the Case Group, these participants each received a tailored “report card” 
evaluating his or her driving performance. An analysis was done of the subsequent self-reported 
driving history for these participants during the first year of this study in the context of their 
report card scores. The specific categories of evaluation used for the report card are summarized 
in Table 16. 
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Table 16. Report Card Evaluation Categories. 


Category Description 
A Shows driver-vehicle readiness 
B Keeps car in balance 
C Accurately positions vehicle 
D Establishes clear path 
E Handles “LOS/POT” blockage 


Controls intersections 
Controls rear zone 

Controls curve 

Controls vehicle in front 
Controls emergency situation 


Spe) FOC) | 


Instructors rated the participants in each category on a scale of 1 to 3, with 1 representing 
satisfactory performance and 3 representing a need for improvement. The distribution of report 
card scores is shown in Figure 12. 


Report Card Score Distribution 
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Report Card Category 


Figure 12. Report Card Score Distribution. 


A Poisson regression approach was taken to determine possible relationships between response 
and explanatory variables (note again that only data from 2006 was analyzed). The goal in fitting 
these models was to identify those report card categories that did a good job of predicting how a 
young driver would perform a year after training. This provided insight into whether different 
predictors worked better for some responses than others, and what combination of predictors 
worked the best overall. Poisson regression is a standard generalized linear model approach that 
models a count response variable as a function of several explanatory variables. By assuming 
that the response variable is Poisson, the probability of a given value of the response variable can 
effectively be expressed as follows: 
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e Huy 
y! 


PY =y)= Equation 2 


Where: 


Y={05 1.2534} 
u = the parameter of the Poisson process 


A unique and important property of Poisson random variables is that the mean response value 
should be equal to the variance of the responses. Since the response variables examined here are 
all Poisson, in order to model the responses as linear functions of the explanatory variables, the 
model follows the following equation: 


log(u(y)) = BO + Bl x1 + B2 x2+...+ Bk xk Equation 3 
Where: 


Lt = mean response value 

y = {0, 1, 2, ...} 

B = coefficient estimates 

x = observed covariate values for a particular case 


The implication of this model is that the response variable has the following distribution: 
Yi ~ Poi(exp(B0 + B1 x1 + B2 x2 +...+ Bk xk )) Equation 4 
Where: 


Y = {0, 1, 2, ...} 
B = coefficient estimates 
x = observed covariate values for a particular case 


The explanatory variables are considered to be ordinal in this case, and thus representative of a 
continuous latent variable representing performance. Further, it was assumed that an increase of 
a point at one region of the report card has the same effect on the response as an increase of a 
point at a different region of the report card. Parameters are estimated via the maximum 
likelihood method, yielding estimates that are approximately unbiased for large sample sizes, and 
standard error estimates rely on sufficient sample size. 


A preliminary fit done with this model revealed some over dispersion of each of the Poisson 
models (which is to say, the mean response value was somewhat lower than the variance of 
responses in the data). This problem was accounted for by fitting a zero-inflated Poisson model 
to the data. A zero-inflated model recognizes that some subset of the population of interest is 
never “at risk”; that is, for example, they never receive a traffic citation. Individuals are “at risk” 
or not, and can be thought of as being separated by some binomial process. Individuals who are 
not “at risk” get response values of 0. The number of traffic citations received by the remainder 
of the population draws from a Poisson random variable. These two processes (the “risk” 
process, which is binary, and the citation counts process, which is Poisson) are modeled in 
tandem, both as functions of the predictor variables (in this case, the score report cards). The 
models used here were fit in the statistical computing environment R (Institute for Statistics and 
Mathematics). 


The following four count response variables were modeled separately: Near Misses, DMV 
Citations, Single-Vehicle Crashes and Multiple-Vehicle Crashes. The explanatory variable suite 


Western Transportation Institute Page 30 


Effects of Defensive Vehicle Handling Training: Phase 3 Results 


consisted of the 10 different score reports, labeled A through J as shown in Table 16. Preliminary 
models were fit that included all explanatory variables as predictors for the Poisson processes. 
Certain covariates were excluded based on insignificance in either process. Each response and 
the covariates used to model its Poisson process are listed in Table 17. 


Table 17. Response Variables and Corresponding Reduced Models. 


Response Model Fit 

Near Miss log (uNear Miss) = B0 + B1 B+ B3 G+ B41 

DMV Citations log (uDMV Citations) = B0 + B2 B+ B8 H 

Single log (uSingle) = B0 + B1 A+B2B+B3D+ B4E+ B51 
Multiple log (uMultiple) = BO + B4 F 


Where the Response is each particular event type response variable found in Table 16 and Model 
Fit is the statistical model containing those predictors found to be significant. 


Over dispersion was problematic in all of these options (see 


Table 18 for means and variances for each response variable below). Model fit was much 
improved by fitting zero-inflated models. 


Table 18. Means and Variances for each Response Variable. 


Variable Mean Response 
Near Misses 0.6713 1.039 
DMV Citations 0.2344 0.2911 
Single-Vehicle Crashes 0.1748 0.2720 
Multiple-Vehicle Crashes 0.1888 0.2387 


Table 19 contains response variable, significant predictors for the Poisson process, predictor 
coefficient estimates, and p-values from Wald’s tests. 
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Table 19. Estimates and P-Values for Significant Predictors in Models for Each Response. 


Significant 
Response Predictors B Estimate ep P-value 
Intercept 3.505 6.3281 .000456 
Keeps Car 
Near Misses Balance -.3713 0.689768 0.035854 
Controls Rear -.3947 0.6738822 0.031907 
Controls Front -.5138 0.598218 0.020991 
DMV Citations pre autes -.9262 0.3960559 0.0153 
Balance 
Controls Curve .6712 1.956584 0.0180 
Intercept -9.5484 7.131528e-05 < .00001 
Vehicle Readiness 1.6904 5.421649 0.00637 
Keeps Car 
Single-Vehicle Balance 1.3997 4.053984 0.00048 
Crashes Clear Path -.7203 0.4866063 0.03003 
Handles 
LOS/POT 1.0782 2.939384 0.00671 
Controls Front 1.7990 6.043601 < .00001 
Multiple-Vehicle Controls 0.74628 2.109139 0.045 
Crashes Intersections 


Where Response is the event type response variable found in Table 16, Significant Predictors is the list of all 
predictors found to be significant in the model, £ Estimate is the predictor coefficient estimates, ef is the predictor 
coefficient estimate multiplied with e, and P-value is the level of significance (all p-values reported here are 
significant with an a < 0.05. 


Coefficient interpretation is contingent on all other covariates being included in the model. The 
interpretation of the coefficients is that for each one-unit increase in score report, the response 
value is expected to multiple by eB. Thus, for a one-unit increase in score report Keeps Car in 
Balance, we expect the number of near misses to be 0.69 times the number of near misses at the 
lower score on score report Keeps Car in Balance. In summary, for Near Misses, Keeps Car in 
Balance, Controls Rear Zone, and Controls Front are significant predictors. For DMV Citations, 
after controlling for Zero-inflation, Keeps Car in Balance and Controls Curve are significant. For 
Single Vehicle Crashes after controlling for Zero-inflation Vehicle Readiness, Keeps Car in 
Balance, Clear Path, Handles LOS/POT, and Controls Front are all significant predictors. For 
Multiple Vehicle Crashes, Controls Intersections is a significant predictor in the Zero-inflated 
model. 


4.5. Summary of Results 


A summary of the key results from each of the dependent variables measured over the entire 
study period is presented in Table 20. 
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Table 20. Summary of Results over All Survey Years 


Summary of Differences: Case versus Control Groups 


Survey Adjusted for Exposure Unadjusted for Exposure 
Year 


Case group has significantly lower self-reported 


me No Significant citation rates than control group 


Differences, Case group has significantly lower self-reported 


sia Case Versus Control citation rates than control group 


Group Case group has significantly higher involvement in 


2008 single-vehicle crashes 


2009 No significant differences between groups 


Summary of Differences: Changes Within Each Group, Over Time 


Group Adjusted for Exposure Unadjusted for Exposure 


Self-reported citations significantly increased 2006— 
2007. 

Self-reported citations marginally significantly 
decreased 2007—-2009. 

Near miss involvement significantly reduced every 
year. 

Single-vehicle crashes reduced 2008-2009, not 
significant. 

Multiple-vehicle crash involvement significantly 
decreased 2007-2009. 


No Significant Differences 
within Case Group Over 
Time 


Case 
Group 


Self-reported citations significantly increased 2006— 

2007, significantly decreased 2007-2009. 

Near miss incident involvement significantly reduced 

between 2006 and 2009, reductions are shown in 2007 
and 2008 as well but significance is not shown. 

Self-reported citations significantly increased 2006— 

2007. 

Single-vehicle crash involvement significantly reduced 

from 2008-2009. 

Multiple-vehicle crashes show significant reduction 

from 2007-2008, than an insignificant increase 2008— 

2009. 


No Significant Differences 
within Control Group 
Over Time 


Control 
Group 
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Summary of Gender Effects 


Category Adjusted for Exposure Unadjusted for Exposure 
Males have higher exposure than females in 
Exposure No Significant Effect of Gender | all years, significant only in 2006, 2007, and 
2009. 
Self-reported _ Males have higher involvement in self- 
tates No Significant Effect of Gender | reported citations in all years, significant 
only in 2006 and 2007. 
Males have higher DMV Males have higher involvement in DMV 
DMV ee : uae nes 
Gadices citations than females in all citations in all years, significant in 2006, 
years, but not significant marginally significant in 2008. 


Males show higher involvement in single- 


ming events No Significant Effect of Gender | vehicle crashes in all years, however not 


Crashes ae 

significant. 
Multiple- Females show higher involvement in 
vehicle No Significant Effect of Gender | multiple-vehicle crashes in all years, 
crashes however not significant. 


Males show higher involvement in total 


Total crashes | No Significant Effect of Gender ; eae 
crashes in all years, however not significant. 


Skid Car Training Scores 


Post-training instructor scores were significantly higher than pre-training scores. 


Report Card Scores 


Response Variable Significant Predictors 
Nese vig Keeps Car Balance, Controls Rear, Controls 
Front 
DMV Citation Keeps Car Balance, Controls Curve 


Vehicle Readiness, Keeps Car Balance, Clear 
Path, Handles LOS/POT , Controls Front 
Multiple-Vehicle Crash Involvement Controls Intersections 


Single-Vehicle Crash Involvement 
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5. DISCUSSION AND CONCLUSIONS 


This report summarizes the descriptive statistics and statistical analysis for the self-reported and 
DMV records pertaining to safety-relevant driving performance of one group of teen drivers that 
received additional driving training (Case Group) compared to a second group that did not 
experience any supplemental training (Control Group). Aggregating the 2006, 2007, 2008, and 
2009 data, the following are the statistically significant findings relating to the Case Group 
versus the Control Group: 


e In 2006 and 2007, there was a significantly lower number of self-reported 
citations in the Case Group compared to the Control Group; however, when 
adjusted for driving exposure the differences were not found to be 
significant. 

e When adjusted for driving exposure, the citation, near-miss and collision 
experience between the two groups was not significantly different during any 
of the survey years. 

e The participants’ DMV records for non-driving and driving-related citations 
did not correlate with the self-reported citation data. This was discovered to 
be due to counting of parking citations in the self-reported data, which were 
not considered reportable in the DMV database. 

e The report card data was demonstrated to be a worthy instrument to predict 
“at risk” drivers, in that: 

= for near misses, Keeps car in balance, Controls rear zone, and Controls 
vehicle in front are significant predictors; 

« for DMV citations, Keeps car in balance and Controls curve is 
significant; 

« for single-vehicle collisions, Shows driver-vehicle readiness, Keeps 
car in balance, Establishes clear path, Handles “LOS/POT” blockage, 
and Controls vehicle in front are all significant predictors; 

* for multiple-vehicle collisions, Controls intersections is a significant 
predictor; and 

« for likelihood of receiving a citation and involvement in near misses 
and single-vehicle collisions, considered collectively, the factor Keeps 
car in balance appears to be the most reliable predictor. 


The statistical finding that the Case Group recorded fewer citations in 2006 and 2007 does 
suggest some potential safety benefits for the supplemental driver-training course evaluated in 
this study. However, the absence of a significant difference in the collision experience of the two 
groups during this same period makes it unclear whether the training resulted in an immediate 
safety benefit. Furthermore, once exposure to driving was accounted for, the differences between 
the Case and Control Groups were no longer significant. It should be noted that other possible 
positive outcomes of the training, such as increased knowledge, increased skills, and increased 
driving adaptability, cannot be measured by analyzing the dependent variables collected in the 
driver survey. The short-term effects of the training were observable in the analysis of the pre- 
training and post-training test scores, indicating an immediate transfer of skills from instruction 
received regarding vehicle dynamics, balance, and control. Relative to the longer term impact of 
the training, the absence of a statistical difference in self-reported citations for the two groups in 
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2008 may indicate that the effects of the training are no longer noticeable three years after the 
training takes place. 


The inconsistencies between the subjective self-reported citations and the objective numbers of 
DMV citations were discovered to be due to the inclusion of parking citations in the self-reported 
data, which were not considered reportable in the DMV database. Notably in this case, the 
differences in the two datasets affect the conclusion reached regarding differences in citation 
experience between the Case and Control Groups. This outcome supports the need to incorporate 
both types of data collection methods. By way of example, similar to the findings in research by 
Schultheis et al. (2002), the DMV records under represent driving behaviors and experiences 
because these records do not contain information on unsafe driving behaviors. Furthermore, the 
participants’ understanding of citations may have included what they felt were unsafe driving 
behaviors associated with the incident for which they were cited. Also, they may have simply 
lost track of the year in which they received the citation. 


Future research efforts of this type could benefit from certain variations in the study design. An 
ideal method of measuring training effectiveness is through naturalistic driver observation; this 
approach allows for direct observation of the driver’s ability to, for example, handle a vehicle 
safely when the vehicle is skidding. Alternative future design suggestions include phone 
interviews, which permit the collection of detailed information necessary to complete a more 
comprehensive analysis. Conducting surveys at six-month intervals, at least initially, instead of 
using a one-year data collection period may help to better capture the length of time before the 
training benefit is no longer discernable. This six-month interval would coincide better with the 
well documented issue of higher fatality rates within six months of licensure. If surveys are to be 
used, a redesigned survey in a more standardized format similar to crash databases would be 
desirable. By way of example, using an exposure metric such as Vehicle Miles Traveled (VMT) 
may be more useful than a categorical “hours driven per week.” Additionally, the surveys used in 
this research included a request that the participant briefly describe each encountered event. 
Responses ranged from complex discussions with diagrams and costs associated with repair, to 
less informative descriptions that avoided any discussion of severity, fault, or type of collision. 
Redesigning the survey to ensure collection of severity (e.g., vehicle damage only, injury, 
fatality) and crash type (e.g., angular, rear end, head on, sideswipe) information may aid in 
analysis of training effects and observing participant recollection and application of driving 
behaviors. A follow-up study using police crash reports that record specific crash information for 
each participant would be beneficial, and could achieve the same effect as a better designed 
survey. 
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APPENDIX A: YOUNG DRIVER SURVEY 


YOUNG DRIVER SURVEY 

Name (as appears on driver license) Driver License # 
Address City/State Zip 
Gender: M F __ Date of Birth How long have you been driving? Years Months 
1. What city/school did you receive driver education? 
DZ How many hours a week do you usually drive? Check one: Under2_— 3-5___—6-10__—‘i11-15 

16-20 ; more than 20 
3. How often do you have passengers in your vehicle? 

Check one: Daily Weekly Seldom 

Are passengers usually (check all that apply) 

family non-family teens adults 


On average how many passengers each trip? 


4. What type(s) of vehicles do you usually drive? Check ones that apply: 
Car: Small Medium__ Large 
SUV: Small Medium__ Large 
Pickup: Small Medium__ Large 
Other Describe 
ay What hours of the day do you usually drive? Check ones that apply: 6am—noon __; 


Noon-—6pm__; 6pm—9 pm__; 9pm-—mid-night__; Mid-night- 6am___ 


6. In the past year have you received any of the following legal citations; if so how many? 
Moving violations (tickets) ; Moving warnings ; MIPs 
DUIs ; Suspended license 
Other Describe 


ci In the past year, have you had any near miss crashes; if so how many? 
Describe your near misses, if any. 


8. In the past year have you had any single vehicle crashes (yours was the only vehicle involved), such as running off the 
road? Ifso, how many? 
Briefly describe: 

9. In the past year have you had any multiple vehicle crashes (yours was not the only vehicle involved); if so how many? 


List and briefly describe the crashes, ifany 


Please complete and return this survey by August 15, 2006 and we will send you $20. 
Western Transportation Institute Montana State University Bozeman, MT 59717-4250 
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